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ABSTRACT 

The ARCS Motivation Theory was proposed to guide instructional designers and 
teachers who develop their own instruction to integrate motivational design strategies 
into the instruction. There is a lack of literature supporting the idea that instruction for 
blended courses if designed based on the ARCS Motivation Theory provides different 
experiences for learners in terms of motivation than instruction developed following 
the standard instructional design procedure for blended courses. 

This study was conducted to compare the students' motivational evaluation of blended 
course modules developed based on the ARCS Motivation Theory and students' 
motivational evaluation of blended course modules developed following the standard 
instructional design procedure. Randomly assigned fifty junior undergraduate students 
studying at the department of Turkish Language and Literature participated in the 
study. Motivation Measure for the Blended Course Instruction (MMBCI) instrument 
was used to collect data for the study after the Confirmatory Factor Analysis (CFA). 
Results of the study indicated that designing instruction in blended courses based on 
the ARCS Motivation Theory provides more motivational benefits for students and 
consequently contributes student learning. 

Keywords: Blended learning; ARCS motivation theory; instructional design 
INTRODUCTION 

The widespread availability of digital learning technologies increased the integration 
of computer-mediated instructional elements into the traditional classrooms (Bonk & 
Graham, 2005). Learning systems benefiting from technology are being developed at 
ever increasing rates (Keller, 2008). Changing learning needs of students and 
developments in technology are forcing educators to integrate technology in the 
teaching and learning to maximize student learning. Many universities offer distance 
courses to respond to the diverse needs of today's learners (Akdemir & Koszalka, 
2008). Combining the benefits of face-to-face courses and distance courses, blended 
courses are used to deliver the instruction in more than one delivery format. It is 
imperative for educators to understand how to design and develop effective blended 
learning experiences that incorporate both face-to-face and computer-mediated 
elements (Bonk & Graham, 2005). Designing instruction for blended courses is a 
complicated process since increased number of factors involves in the equation. 


73 



Systematically designed instruction has been proven to affect student learning greatly 
(Gagne, Briggs & Wager, 1992). Incorporating systematic design, instructional design 
is concerned with understanding, improving, and applying methods of instruction 
(Reigeluth, 1983). Utilizing from instructional design theories and models, effective 
instruction can be designed and developed for blended courses. However even when 
prepared according to sound instructional design principles, instruction often does not 
stimulate students' motivation (Visser & Keller, 1990). It was pointed out that often 
"motivation" is neglected in the instructional design process (Keller, 1983). More 
importantly online part of the instruction in blended courses contains numerous 
motivational challenges (Keller, 1999). Combining face-to-face and online delivery 
systems, blended learning systems offer opportunities to integrate motivational 
support strategies in novel ways (Keller, 2008). It is abundantly clear that the 
environment can have a strong impact on motivation (Keller, 1999). Instructional 
designers must know how to integrate motivational methods and models into a variety 
of instructional situation (Keller & Litchfield, 2002). 

Integrating motivation in the blended courses is a challenging task for instructional 
designers. The ARCS Motivation Theory was proposed to guide instructional designers 
and teachers who develop their own instruction to integrate motivational design 
strategies into the instruction. The ARCS model is a model for instructional design 
developed to enhance learner motivation (Capshew, 2005) and has been applied to 
courseware design (Suzuki, Nishibuchi, Yamamoto & Keller, 2004). ARCS Motivation 
Theory has four components. Attention, Relevance, Confidence and Satisfaction are 
the four conceptual components of the theory. Attention category refers to gaining 
learners' attention and sustaining active engagement of learners (Keller, 2008). 
Relevance category includes strategies that establish connections between 
instructional environment and past experiences of learners (Keller, 2008). Confidence 
category incorporates students' feelings and expectancy for success (Keller, 2008). 
The last category satisfaction includes strategies that help learners establish positive 
feelings about their learning experiences (Keller, 2008). Researchers have investigated 
the effectiveness of ARCS model in different learning environments. Song and Keller 
(2001) investigated the effects of a motivationally adaptive computer-assisted 
instruction (CAI) developed in accordance with systematic motivational design 
principles as represented in the ARCS model with fifty nine tenth-grade students. 
Three types of CAIs, motivationally saturated, adaptive, and minimized, were 
developed and completed by tenth-grade students. Motivation, effectiveness, 
continuing motivation, and efficiency were measured and compared in three types of 
CAIs. 

Findings indicated that the ARCS model can be applied effectively to the design of 
motivationally adaptive CAI. Suzuki, Nishibuchi, Yamamoto & Keller, (2004) developed 
a web tool that allowed users to check and revise to improve their instruction based on 
the results of the user reaction survey. The web tool is capable of analyzing the 
questionnaire data and suggests strategies based on the ARCS motivation design to 
improve instruction. Users reported that web tool helped them to think motivational 
enhancement systematically in their instruction. In a single case study, Visser & Keller 
(1990) investigated the effects of motivational intervention developed based on the 
process outlined in the ARCS model of motivational design on participants' attitude 
and performance. Fifteen adult students attending in a staff development course 
participated in the study. 

Results indicated that motivational messages designed based on the ARCS model 
positively affect students' attitude, performance, and consequentially their motivation 
to learn. Shellnut, Savage, and Knowlton (1999) reported the experiences of the 
multimedia course design team in applying the ARCS model to the process of 
designing, developing, and evaluating Computer-based Instruction (CBI). 
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The design team reported that incorporating ARCS model into the basic design will 
continue to provide a solid foundation in the design and development of CBI. 

Studies conducted in face-to-face, computer-based, computer-assisted, and online 
environment demonstrated the benefits of utilizing ARCS model in motivation and 
learning. However there is a lack of literature supporting the idea that instruction for 
blended courses if designed based on the ARCS Motivation Theory provides different 
experiences for learners in terms of motivation than instruction developed following 
the standard instructional design procedure for blended courses. Therefore this study 
was designed to address the following research questions: 

> How does students' evaluation of the instructional modules for the 
components of the ARCS Motivation Theory change in blended course 
modules? 

> Is there a difference in students' evaluation of attention between 
instructional modules developed based on the ARCS model and instructional 
modules developed following the standard instructional design procedure? 

> Is there a difference in students' evaluation of relevance between 
instructional modules developed based on the ARCS model and instructional 
modules developed following the standard instructional design procedure? 

> Is there a difference in students' evaluation of confidence between 
instructional modules developed based on the ARCS model and instructional 
modules developed following the standard instructional design procedure? 

> Is there a difference in students' evaluation of satisfaction between 
instructional modules developed based on the ARCS model and instructional 
modules developed following the standard instructional design procedure? 

METHOD 

Instructional Context 

The study was conducted in a three-credit Instructional Technology and Material 
Development course offered at the department of Turkish Language and Literature of a 
public university located in the BlackSea region of Turkey in the Fall 2007 semester. 
The Moodle has been used to teach online part of the blended courses since 2006 in 
the university where the study was conducted. The compulsory Instructional 
Technology and Material Development course was divided in two sections. 

Participants completed two hours theory and two hours practice sections of the course 
each week. Students were expected to learn basic processes of instructional design in 
the theory part of the course and develop an instructional material in their field of 
study in the practice part of the course. 

Participants 

The participants were junior undergraduate students enrolled at the four year public 
college. Sixty-one undergraduate students enrolling in a three-credit Instructional 
Technology and Material Development course participated in the study. Two sections 
of the course were taught to participants in the same semester by the same instructor. 
Students randomly selected one of these sections when registering in the course. Then 
researchers randomly assigned one section of the course as the experimental which 
had 32 registered students and the other section of the course which had 29 students 
as the control group. Then researchers identified 25 students in the experimental 
group and 25 students in the control group each as the study participants for data 
collection purposes in the six week period due to the absence of students in various 
parts of the study. 38% percent of the participants were male and 62% of the 
participants were female. 
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Instrument 

Motivation Measure for the Blended Course Instruction (MMBCI) instrument developed 
by Akdemir & Colakoglu (2008) was primarily used for data collection. The MMBCI 
instrument was used to measure the motivational evaluation of instructional 
materials. Akdemir & Colakoglu (2008) used explanatory factor analysis when 
developing the instrument. They identified four factors namely attention, relevance, 
confidence and satisfaction which comply with four categories defined in the ARCS 
motivation model developed by Keller (1983). However, Exploratory Factor Analysis 
(EFA) is a method to examine underlying construction of the measured variables and 
reduce a large number of observed variables into the small number of factors 
(Tabachnick & Fidell, 1989). EFA is generally used as a theory generating rather than a 
theory testing procedure. In contrast to EFA, Confirmatory Factor Analysis (CFA) is 
used for the strong theoretical or empirical foundation that permits researchers 
examine their exact factor model to their observed data (Stevens, 1996). 

The Confirmatory Factor Analysis was used with the sample used in the study in order 
to test the model fitness of the MMBCI instrument with ARCS model's four-factor 
structure. AMOS 7.0 statistical software package was used to conduct the 
Confirmatory Factor Analysis. 

The first assumption of CFA is multivariate normality of the variables. Multivariate 
normality of the variables in the model is necessary to provide well-behaved analysis 
for the researchers (Dilalla, 2000). Also multivariate normality should be ensured for 
maximum likelihood fitting function. 

The multivariate normality of the variables is presented at the Table. 1 Critical ratio of 
the both skewness and kurtosis for items of the instrument did not exceed the range 
from -2 to 2 except for the Item-10 and Item-4. Nevertheless multivariate normality of 
the all variables' critical ratio was 1.683 which is an acceptable value. Therefore 
maximum likelihood function could be put into operation for the given data. 


Table: 1 

Multivariate Normality of the Variables 


Variable 

Min 

Max 

Skewness 

C.R. 

Kurtosis 

C.R. 

V2 

1 

5 

-0.152 

-0.483 

-1.196 

-1.906 

V3 

1 

5 

-0.023 

-0.074 

-1.368 

-2.181 

V4 

1 

5 

-0.861 

-2.745 

-0.177 

-0.282 

V5 

2 

5 

-0.552 

-1.761 

0.278 

0.444 

V6 

1 

5 

-0.414 

-1.321 

0.023 

0.036 

V9 

1 

5 

-0.343 

-1.094 

-1.042 

-1.661 

V10 

1 

5 

-1.091 

-3.477 

1.427 

2.276 

Vll 

1 

5 

-0.497 

-1.585 

-0.543 

-0.865 

V12 

1 

5 

-0.641 

-2.045 

1.22 

1.945 

V13 

2 

5 

-0.225 

-0.717 

-0.259 

-0.413 

V15 

1 

5 

-0.555 

-1.768 

-0.16 

-0.256 

Multivariate 





7.286 

1.683 


The second assumption for CFA is the sample size. In this study a model would not be 
generalized. Only the model fitness was tested for the specific sample to determine the 
trends of the components in ARCS Motivation Theory in the six-week period of the 
course. Therefore the first hypothesis was used to test the model fitness for the 
sample used in the study. 
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The study sample was obtained from two groups. One of them was control with 29 
students and the other was experimental group with 32 students. Then the maximum 
likelihood function could be put into operation for the given data. In CFA models, there 
are three important points for avoiding misspecification: 

> The wrong number of factors, 

> The wrong pattern of loadings, and 

> Unmodeled sub-factors. 

Table: 2 Component Correlation Matrix 


Component 

Satisfactio 

n 

Attend 0 

n 

Confidenc 

e 

Relevance 

Satisfaction 

1.000 

.023 

.178 

.268 

Attention 

.023 

1.000 

.040 

.102 

Confidence 

.178 

.040 

1.000 

.156 

Relevance 

.268 

.102 

.156 

1.000 


Note: Extraction Method: Principal Component Analysis, Rotation Method: 
Oblimin with Kaiser Normalization 


Hoyle (2000) suggested that using EFA solutions for CFA is good starting point for the 
researchers so the first model was chosen based on Principal Component Analysis 
solutions in this study. 

The factor structure of the instrument for both oblique and varimax rotation was 
implemented to determine the component correlation matrix. The Principal Component 
Analysis (PCA) was conducted on the data and the 4-factor and 11-item were left in 
the instrument after the analysis. Then the 4-factor model of the instrument was 
designed in AMOS. After designing, the component correlation matrix was used to 
determine the correlations among components in the CFA model. 


Then maximum likelihood estimation method was applied. Probability levels of all 
models were found not significant. This indicated that the covariance matrix and 
population covariance matrixes were not statistically different from each other. 

Therefore p=0.179 leads us to reach the conclusion that we fail to reject the null 
hypothesis. Chi-square statistic was used in this study for model comparison. The 
lowest A 2 value was obtained for the Model-3. Therefore the Model-3 was preferred 
over other three models. 


Table: 3 

Table for Fit Indices of Three Models 



X 2 

P 

D 

f 

RMSE 

A 

Pclos 

e 

RMR 

GFI 

CFI 

TLI 

NFI 

AIC 

Model- 

49.49 

0.14 

4 



0.14 

0.87 

0.97 

0.96 

0.89 

101.49 

1 

2 

4 

0 

0.063 

0.342 

2 

5 

6 

8 

2 

2 

Model- 

48.30 

0.14 

3 



0.12 

0.87 

0.97 

0.96 

0.89 

102.30 

2 

8 

6 

9 

0.063 

0.341 

9 

8 

7 

7 

4 

8 

Model- 

45.85 

0.17 

3 



0.06 

0.88 

0.98 

0.97 

0.90 

101.85 

3 

3 

9 

8 

0.059 

0.387 

8 

3 

0 

2 

0 

3 


Note: (RMSEA): Root Mean Square Error of Approximation; (GFI): Goodness of Fit Index; (TLI): 
Tucker-Lewis Index; (NFI): Normed Fit Index; (RMSR): Root Mean Square Residual; (CFI): 
Comparative Fit Index; (AIC): Akaike Information Criterion 


When the Table.3 is examined the chi-square values are decreasing from the Model -77 
1 to the Model-3. Correspondingly the probability levels of the models are 



increasing which makes the Model-3 better than other models. Also other fit indices' 
values (GFI, CFI, TLI and NFI) are increasing as well. 


This is evident that the Model-3 is better than other models. Also the RMSEA value of 
the Model-3 is less than .06 which indicates that it is a well-fitting model (Kim, & 
Bentler, 2006). 

Also the Pclose value of the RMSEA indicates that the RMSEA value is no greater than 
.05. Found RMSEA value is very close to the RMSEA value, .054, obtained in a study 
conducted to validate the Keller's IMMS survey (Huang, Huang, Diefes-Dux & Imbrie, 
2006). 

In summary, the Model-3 that was presented in the Figure: 1 was chosen over other 
models for the study. All four latent variables which were described in the ARCS model 
were also present in the Model-3 and they were correlated with each other. 



Figure. 1 

Path Diagram of Model-3. 

Note: A: Attention, R: Relevance, C: Confidence, S: Satisfaction 


Intervention 

Two different types of instruction were developed for the online part of the blended 
course. The experimental group (25 students) received instruction that was designed 
based on the ARCS Motivation Theory. Keller (1987) recommended strategies for 
designing instruction based on the ARCS Motivation Theory. 
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These strategies and examples used in the blended course modules were presented at 
the Tables 4 for the attention category, at the Table-5 for the relevance category, at 
the Table-6 for the confidence category, and at the Table-7 for the satisfaction 
category. The blended course modules developed for the control group (25 students) 
included only the presentation of the course content. Figures, tables and graphs were 
also used in the course modules of the control group. 

Table: 4 

Attention Strategies Used in Blended Course Modules 



Strategies Recommended 
for the Attention 

Category by Keller (1987) 

Examples used in the 

Blended Course Modules 

Use surprise or 

Perceptual uncertainly to gain 

Arousal interest. 

Cartoons and movies were 
used at the beginnig of the 
course modules 

Inquiry 

Stimulate curiosity by 
posing challenging 
questions or problems to 
be solved. 

Challenging questions 
were asked to sudents in 
the modules. 

Variability 

Sustain interest with 
variations in presentation 
style, human interest 
examples and 
unexpected events. 

Themes and styles used in 
the blended course 
modules were changed in 
modules. 

Table: 5 

Relevance Strategies Used in Blended Course Modules 


Strategies Recommended for the 
Relevance Category by Keller (1987) 

Examples used in the 
Blended Course 

Modules 

Goal 

Orientatio 

n 

Provide statements or examples of 
the utility of the instruction or 
present goals. 

The objectives of the 
course were presented 
in the course modules. 

Motive 

Matching 

Make instruction responsive to 
learner motives and values by 
providing personal achievement 
opportunities, cooperative activities, 
leadership responsibilities and 
positive role models. 

Questions making 
conections between 
objectives and 
students' future work 
were directed to 
students. 

Famiiiarit 

y 

Make the materials and concepts 
familiar by providing concrete 
examples and analogies related to 
learner's work. 

Concrete examples 
were presented to 
students. 
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Table: 6 

Confidence Strategies Used in Blended Course Modules 



Strategies 

Recommended for the 
Confidence Category by 
Keller (1987) 

Examples used in the 
Blended Course Modules 

Learning 

Requirements 

Establish trust and 
positive expectations 
by explaining the 
requirements for 
success and the 
evaluative criteria. 

Evalaution criteria was 
presented at the course 
syslabus 

Success 

Opportunities 

Increase belief in 
competence by 
providing many, varied 
and challenging 
experinces which 
increase learning 
success. 

Challenging questions 
were directed to the 
students in the 
discusion board. 

Personal 

Control 

Use techniques that 
offer personal control 
and provide feedback. 

Learners completed 
each module in their 
learning phase. 

Feedbacks were 
provided after the 
exercises. 

Table: 7 

Satisfaction Strategies Used in Blended Course Modules 


Strategies Recommended 
for the Satisfaction 
Category by Keller (1987) 

Examples used in the 
Blended Course 

Modules 

Natural 

Consequenc 

es 

Provide problems, 
simulations, or work 
samples that allow 
students to see how they 
can now "real-world" 
problems. 

Real problem scnerios 
were presented and 
students developed 
solutions for problems. 

Positive 

Consequenc 

es 

Use verbal praise, real or 
symbolic rewards, and 
incentives. 

After each expercise 
symbolic rewards 
were presented. 

Equity 

Make performance Course exams were 

requirements consistent covered all the areas 

with stated expectations, taught in the blended 

and provide consistent course modules. 

measurement standards for 
all learners' task. 


Procedure 

Total of twelve course modules, six for the experiment and six for the control group, 
were developed for the study. Experimental group and the control group completed 
each module in one-week. The study was completed in six weeks. 












Motivation Measure for the Blended Course Instruction instrument, which was 
confirmed after the Confirmatory Factor Analysis, was used to collect data from the 
experiment and control groups at the end of the each course module. 

ANALYSIS 

All items of the instrument were reviewed for any missing data or error. No problem 
was detected. Participants rated the each statement in the instrument ranging from 
1= Strongly Disagree to 5=Strongly Agree. Participants' responses for the Attention, 
Relevance, Confidence, and Satisfaction categories were entered into the SPSS Version 
13 for further analysis. Descriptive analysis and t-test for independent samples were 
conducted to investigate research questions. 

RESULTS 

The first research question investigated students' evaluation of the instructional 
modules for the components of the ARCS Motivation Theory in blended course modules 
in the six week period. Students' evaluation of the instructional modules developed 
based on the ARCS Motivation Theory (experimental treatment) indicated that 
modules developed for the experimental group took more attention of the students 
(see Figure 2, 3, 4, 5). Also, students' attention increased as the study progresses. 
Similarly students' evaluation of the instruction showed that students' responses for 
the relevance category progressively increased in the first four weeks of the study and 
then slightly decreased in the fifth and sixth week of the study. However students' 
responses were still higher in the last two week of the study than the initial week for 
the relevance category. Students' evaluation of the instruction revealed a steadily 
increase in the confidence from the first week of the study until the last week. 

Finally students' satisfaction decreased in the second week but then increased 
progressively. However a slight decrease was observed for the students' satisfaction in 
the last week of the study. On the other hand, students' evaluation of the instructional 
modules developed following the standard instructional design procedure (non 
treatment) was different from students' evaluation of the instructional modules 
developed based on the ARCS Motivation Theory. Findings revealed that students' 
evaluation of instruction for the attention, relevance, confidence, and satisfaction fell 
below the initial week's evaluation for the ARCS categories (see Figure 2, 3, 4, 5). 
Especially for the attention and relevance categories, the gap gradually increases 
between the initial week's responses and subsequent weeks' responses. The dramatic 
decreases were especially observed in the first weeks of the course. 


Attention (Experiment-Control) 


5.00 

4.50 
4.00 

3.50 
3.00 

2.50 
2.00 

1.50 
1.00 
0.50 
0.00 



1 2 3 4 5 6 


-Exp. 

-Control 


Figure: 2 

Instructional Evaluation for the Attention Category 
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Relevance (Experiment-Control) 



1 2 3 4 5 6 


-Exp. 

-Control 


Figure: 3 

Instructional Evaluation for the Relevance Category 


Confidence (Experiment-Control) 


4.50 
4.00 

3.50 
3.00 

2.50 
2.00 

1.50 
1.00 
0.50 
0.00 



1 2 3 4 5 6 


-Exp. 

-Control 


Figure: 4 

Instructional Evaluation for the Confidence Category 



Figure: 5 

Instructional Evaluation for the Satisfaction Category 
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The second research question investigated whether there is a difference in students' 
evaluation of attention between instructional modules developed based on the ARCS 
model and instructional modules developed following the standard instructional design 
procedure. Results of the t-test for independent samples revealed statistically 
significant differences found in students' responses when students' responses were 
compared in the six-week period for the attention category (Figure. 2 and Table: 8). 
The differences in students' evaluation of the modules between the experimental 
modules and control group's modules were increasing from the first week (Mean 
Difference=1.32) towards the last week (Mean Difference=2.38). Findings also 
revealed that students' evaluations in experimental group's modules increased from 
the first week's module (=4.09) to the last week's module (M=4.29). On the other 
hand students' evaluations in the control group's modules decreased for the attention 
category from the beginning of the study (M=2.77) to the end of the study (M = 1.9). 
The third research question investigated whether there is a difference in students' 
evaluation of relevance between instructional modules developed based on the ARCS 
model and instructional modules developed following the standard instructional design 
procedure. 

Table: 8 

The Comparison of the Attention Category between 
Experimental and Control Group's Evaluation 




Levene's 
Test for 







Equality of 
Variances 










Sig. 

Mean 



F Sig. 

t 

Df 

(2-tailed) 

Difference 


Equal 

variances 

21.14 .00 

a. n 

5.138 

48 

.000 

1.32000 

Week-1 

assumed 

Equal 
variances 
not assumed 

*1 u 

5.138 

30.141 

.000* 

1.32000 


Equal 

00 






variances 

8.687 ■ l l U 

9.197 

48 

.000 

1.69333 

Week-2 

assumed 

Equal 
variances 
not assumed 

5 

9.197 

33.410 

.000* 

1.69333 


Equal 

variances 

3.440 o 7 

11.54 

48 

.000* 

2.00000 

Week-3 

assumed 


4 





Equal 

variances 


11.54 

A 

43.520 

.000 

2.00000 


not assumed 


4 





Equal 

variances 

3.909 ° 5 

11.08 

48 

.000* 

2.05333 

Week-4 

assumed 

4 

3 





Equal 

variances 


11.08 

40.246 

.000 

2.05333 


not assumed 


3 





Equal 

variances 

9.698 °° 

12.04 

8 

48 

.000 

2.12000 

Week-5 

assumed 

Equal 
variances 
not assumed 

3 

36.487 

.000* 

2.12000 

12.04 

8 


Equal 

variances 

4.632 03 

15.16 

n 

48 

.000 

2.38667 

Week-6 

assumed 

o 

y 





Equal 

variances 


15.16 

Q 

39.862 

.000* 

2.38667 


not assumed 







* p< 0.05 
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Results of the t-test for independent samples revealed statistically significant 
differences found in students' responses when students' responses were compared in 
the six-week period for the relevance category (Figure: 3 and Table: 9). The 
differences in students' evaluation of the modules between the experimental modules 
and control group's modules were increasing from the first week (Mean 
Difference=0.06) towards the last week (Mean Difference=0.9). Findings also revealed 
that students' evaluations in experimental group's modules steadily increased in the 
first five weeks of the study. However students' evaluation in the last week was close 
to the first week's evaluation result. On the other hand students' evaluation of the 
modules developed following the standard instructional design procedure dramatically 
decreased from the first week's module (M=3.88) to the last week's module (M=3.04). 

Table: 9 

The Comparison of the Relevance between 
Experimental and Control Group's Evaluation 


Levene's Test 
for Equality 
of Variances 






Sig. 

Mean 





(2- 

Differen 

F 

Sig. 

t 

df 

tailed) 

ce 


Week-1 

Equal variances 
assumed 

3.25 

8 

.077 

.292 

48 

.771* 

.06000 


Equal variances 
not assumed 



.292 

41.53 

7 

.772 

.06000 

Week-2 

Equal variances 
assumed 

3.97 

2 

.052 

1.463 

48 

.150* 

.26000 


Equal variances 
not assumed 



1.463 

39.10 

9 

.152 

.26000 

Week-3 

Equal variances 
assumed 

2.66 

1 

.109 

1.926 

48 

.060* 

.40000 


Equal variances 
not assumed 



1.926 

43.42 

5 

.061 

.40000 

Week-4 

Equal variances 
assumed 

3.85 

6 

.055 

2.279 

48 

.027* 

.44000 


Equal variances 
not assumed 



2.279 

38.15 

2 

.028 

.44000 

Week-5 

Equal variances 
assumed 

2.41 

2 

.127 

5.704 

48 

.000* 

.82000 


Equal variances 
not assumed 



5.704 

44.03 

7 

.000 

.82000 

Week-6 

Equal variances 
assumed 

.819 

.370 

4.585 

48 

.000* 

.90000 


Equal variances 
not assumed 



4.585 

44.17 

8 

.000 

.90000 


* p< 0.05 


The forth research question investigated whether there is a difference in students' 
evaluation of confidence between instructional modules developed based on the ARCS 
model and instructional modules developed following the standard instructional design 
procedure. 
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Results of the t-test for independent samples showed statistically significant 
differences found in students' responses when students' responses were compared in 
the six-week period for the confidence category (Figure: 4 and Table: 10). The 
differences in students' evaluation of the modules between the experimental modules 
and control group's modules were increasing from the first week (Mean 
Difference=1.23) towards the last week (Mean Difference=1.68). Findings also 
indicated that students' evaluation in experimental group's modules slightly but 
steadily increased in the first five weeks of the study. In contrast students' evaluation 
of the modules for the control group slightly decreased from the first week's module 
(M=2.7) to the last week's module (M=2.4). 

Table: 10 

The Comparison of the Confidence Category between 
Experimental and Control Group's Evaluation 


Levene's Test for 
Equality of 
Variances 



F 

Sig. 

t 

df 

Sig. 

(2- 

tailed) 

Mean 

Difference 


Week- 

Equal variances 
assumed 

10.76 

6 

.002 

5.27 

6 

48 

.000 

1.2333 

3 

1 

Equal variances not 
assumed 



5.27 

6 

38.331 

.000* 

1.2333 

3 

Week- 

•y 

Equal variances 
assumed 

2.845 

.098 

6.43 

4 

48 

.000* 

1.3866 

7 

4 

Equal variances not 
assumed 



6.43 

4 

43.873 

.000 

1.3866 

7 

Week- 

•> 

Equal variances 
assumed 

7.223 

.010 

7.70 

3 

48 

.000 

1.5800 

0 

3 

Equal variances not 
assumed 



7.70 

3 

38.229 

.000* 

1.5800 

0 

Week- 

A 

Equal variances 
assumed 

5.769 

.020 

7.01 

2 

48 

.000 

1.5066 

7 


Equal variances not 
assumed 



7.01 

2 

37.810 

.000* 

1.5066 

7 

Week- 

C 

Equal variances 
assumed 

2.699 

.107 

8.67 

1 

48 

.000* 

1.5800 

0 

3 

Equal variances not 
assumed 



8.67 

1 

40.753 

.000 

1.5800 

0 

Week- 

C 

Equal variances 
assumed 

4.685 

.035 

7.59 

7 

48 

.000 

1.6800 

0 

o 

Equal variances not 
assumed 



7.59 

7 

39.449 

.000* 

1.6800 

0 


* p< 0.05 


The last research question investigated whether there is a difference in students' 
evaluation of satisfaction between instructional modules developed based on the ARCS 
model and instructional modules developed following the standard instructional design 
procedure. 

Results of the t-test for independent samples revealed statistically significant 
differences found in students' responses when students' responses were compared in 
the six-week period for the satisfaction category (Figure: 5 and Table 11). The 
differences in students' evaluation of the modules between the experimental modules 
and control group's modules were increasing from the first week (Mean 
Differences.29) towards the last week (Mean Difference=0.56). 
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Findings also revealed that students' evaluations in experimental group's modules 
slightly decreased in the first two weeks of the study and then increased. 

Similar changes were observed for the students' evaluation in the control group. 
Similar trend was also observed in students' evaluation of the modules. 

Students' evaluation of the modules increased from the first week's module (M=3.81) 
until the last week's module (M=3.87) for the experimental group and students' 
evaluation of the modules decreased from the first week's module (M=3.52) until the 
last week's module (M=3.31) 


Table: 11 

The Comparison of the Satisfaction Category between 
Experimental and Control Group's Evaluation 




Levene's Test for 
Equality of Variances 







F 

Sig. 

t 

df 

Sig. 

(2- 

tailed) 

Mean 

Differenc 

e 

Week- 

Equal variances 
assumed 

.041 

.841 

2.059 

48 

.045* 

.29000 


Equal variances 
not assumed 



2.059 

47.540 

.045 

.29000 

Week- 

Equal variances 
assumed 

1.092 

.301 

2.112 

48 

.040* 

.38000 


Equal variances 
not assumed 



2.112 

44.738 

.040 

.38000 

Week- 

*5 

Equal variances 
assumed 

1.418 

.240 

2.059 

48 

.045* 

.39000 


Equal variances 
not assumed 



2.059 

43.580 

.045 

.39000 

Week- 

A 

Equal variances 
assumed 

2.285 

.137 

1.506 

48 

.139* 

.28000 


Equal variances 
not assumed 



1.506 

42.816 

.139 

.28000 

Week- 

Equal variances 
assumed 

1.541 

.220 

3.957 

48 

.000* 

.60000 

3 

Equal variances 
not assumed 



3.957 

46.836 

.000 

.60000 

Week- 

6 

Equal variances 
assumed 

3.741 

.059 

2.963 

48 

.005* 

.56000 

Equal variances 
not assumed 



2.963 

41.279 

.005 

.56000 


* p< 0.05 


DISCUSSION AND CONCLUSION 

Motivation is an academic factor that should be considered in the course design 
(Capshew, 2005). It was reported that instructional design incorporating instructional 
and motivational components are critical to achieve learning goals (Keller, 1999). This 
study investigated the effects of designing instructional modules based on the ARCS 
Motivation Theory in blended courses on students' motivation. Data analysis revealed 
that students' evaluation in blended courses modules developed based on the ARCS 
model and students' evaluation in blended course modules developed using standard 
instructional design process are statistically different for all components of the ARCS 
motivation theory. 
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Students' evaluation of instructional modules for motivation increased when the 
instructional modules for blended courses are designed based on the components of 
the ARCS motivation theory. Students experienced all motivational components of the 
ARCS Motivation theory in the instruction and utilized the benefits of the motivation in 
the instruction for their learning when students completed instructional modules 
developed based on the ARCS Motivation Theory. 

While such improvements in motivation were observed in the experimental group, 
students in the control group which received the blended course instruction developed 
following the standard instructional design process perceived less motivational 
components in the instruction. 

When the instruction for blended courses is designed following the standard 
instructional design procedure and motivational factors are neglected, motivational 
measure of the instruction for all components of the ARCS motivation theory 
decreased. Students in blended courses did not experience the motivation in the 
expected level in blended courses when they complete the instructional modules which 
were not designed based on the ARCS Motivation Theory. 

This study confirmed Keller & Litchfield (2002)'s assertion that even the most accurate 
content and related activities can be ineffective without the systematic incorporation 
of motivation to improve student motivation. Motivation does not only stimulate 
students' effort (Capshew, 2005) but also positively influence learning and 
performance (Keller & Litchfield, 2002). Feng & Tuan (2005) found that using ARCS 
Motivation Theory improves students' level of motivation. Feng & Tuan (2005)'s 
findings support the results of this study. Similarly Chyung (2001) concluded that by 
enhancing motivational appeal of instruction, students perceive the instruction more 
interesting and relevant. Keller (1999) suggested that moivation can be inflenced by 
external events. This study demonstrated that when the blended course modules 
designed based on the startegies recommended in the ARCS Moivation Theory, 
motivation of the students improves. Instructional design is a challenging procedure 
which requires the consideration of all elements of the learning to bring about the 
desired change. Designing instruction for courses where more than one medium is 
used for the delivery is even more complex and important. Integrating motivation into 
the instructional design process improves the quality of the instruction. 

It should be remembered that "one consequence of motivation is to contribute to 
better learning' (Keller, 1983). This study demonstrated that students in blended 
courses gain more learning benefits in terms of motivation when instruction is 
designed based the ARCS Motivation Theory. Similar results were achieved in the face- 
to-face (Visser & Keller, 1990), computer-based (Shellnut, Savage & Knowlton, 1999), 
computer-assisted (Song & Keller, 2001), and instruction developed in online 
environments (Suzuki, Nishibuchi, Yamamoto & Keller, 2004) which utilized the ARCS 
Motivation Theory. Therefore instructional designers and instructors should consider 
the integration of the ARCS Motivation Theory in their design consideration for 
blended courses so that learning environments can be managed to stimulate and 
sustain motivation (Keller, 1999). It is suggested that researchers in the future should 
replicate the study with samples in different context to make the results of this study 
more generalizable. 

Authors note: The earlier version of this study was presented at the ED-MEDIA 2008 
which was held in Vienna, Austria. 
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